Quality of Service Issues for Reinforcement Learning Based Routing Algorithm for Ad-Hoc Networks

Mobile ad-hoc networks are dynamic networks which are decentralized and autonomous in nature. Many routing algorithms have been proposed for these dynamic networks. It is an important problem to model Quality of Service requirements on these types of algorithms which traditionally have certain limitations. To model this scenario we have considered a reinforcement learning algorithm SAMPLE. SAMPLE promises to deal effectively with congestion and under high traffic load. As it is natural for ad-hoc networks to move in groups, we have considered the various group mobility models. The Pursue Mobility Model with its superiormobilitymetrics exhibits better performance. At the data link layer we have considered IEEE 802.11e, a MAC layer which has provisions to support QoS. As mobile ad-hoc networks are constrained by resources like energy and bandwidth, it is imperative for them to cooperate in a reasonably selfish manner. Thus, in this paper we propose cooperation with a moderately punishing algorithm based on game theory. The proposed algorithm in synchronization with SAMPLE yields better results on IEEE 802.11e

A Formal Verification Model for Performance Analysis of Reinforcement Learning Algorithms Applied t o Dynamic Networks

Routing data packets in a dynamic network is a difficult and important problem in computer networks. As the network is dynamic, it is subject to frequent topology changes and is subject to variable link costs due to congestion and bandwidth. Existing shortest path algorithms fail to converge to better solutions under dynamic network conditions. Reinforcement learning algorithms posses better adaptation techniques in dynamic environments. In this paper we apply model based Q-Routing technique for routing in dynamic network. To analyze the correctness of Q-Routing algorithms mathematically, we provide a proof and also implement a SPIN based verification model. We also perform simulation based analysis of Q-Routing for given metrics